Logical Particle Filtering

In this paper, we consider the problem of filtering in relational hidden Markov models. We present a compact representation for such models and an associated logical particle filtering algorithm. Each particle contains a logical formula that describes a set of states. The algorithm updates the formulae as new observations are received. Since a single particle tracks many states, this filter can be more accurate than a traditional particle filter in high dimensional state spaces, as we demonstrate in experiments. Consider an agent operating in a complex environment, made up of an unknown, possibly infinite, number of objects. The agent can take actions and make observations of the state of the world, and it knows a probabilistic model of how the state changes over time as a result of its actions and of how the observations are generated from the states. How can it efficiently estimate the underlying state of the environment? Filtering is the problem of predicting a distribution over the underlying environment state given a history of the agent’s actions and observations. This problem is pervasive in AI: a dialogue system has to estimate the belief state of the user; an office-assistant must track the states and relationships among people, meetings, and projects; a household robot has to track the locations of furniture, the state of the dishes in the dishwasher, and the desires of the humans in its house. At their root, these problems are controlled hidden Markov models (HMMs) or POMDPs. The standard techniques for filtering in such models require enumeration of the individual states of the environment. This quickly becomes intractable, and is impossible in infinite worlds. Particle filtering methods [1] make approximations by representing a small set of likely states. They can be executed online with constant computation per time step and can be used to track arbitrary state spaces, but reliable estimates in large domains require a very large number of particles. This problem can often be ameliorated by using RaoBlackwellization, in which the filtering distribution is decomposed into two factors, one that is sampled and one that is computed exactly. Rao-Blackwellization has been effectively applied in both propositional [2] and relational [3] state representations. In both cases, however, a finite universe of objects must be known in advance. Quantified logical expressions are a powerful method for using short descriptions to name large (possibly infinite) sets of states. When the underlying model of uncertainty in the domain is nondeterministic rather than probabilistic 2 L. S. Zettlemoyer, H. M. Pasula, L. P. Kaelbling (that is, there is a set of environment states that are consistent with the action/observation stream at any point in time), logical expressions can be used as the basis of algorithms [4] that track effectively in large state spaces. However, it is not obvious how to use them in probabilistic tracking problems. Recent work explored MCMC sampling over relational structures [5], lifted probabilistic inference [6], and exact inference in relational HMMs [7] but we do not know of relational sampling techniques for probabilistic filtering. In this paper, we combine the ability of particle filters to focus only on the most likely underlying states with the ability of logical expressions to tractably name large sets of complex states. We develop an online, logical particle filtering algorithm that maintains a set of quantified logical formula or hypotheses, each of which potentially describes a large or infinite set of states. The hypotheses are built up incrementally, making discriminations only as needed to track the results of the agent’s actions and incorporate the information from its observations. Aspects of the environment about which the agent gets no information are not explicitly represented. As a result, we get a robust and efficient filter whose complexity is driven by the information content of the agent’s actions and observations. As a running example, we will consider an idealized robot that needs to map an environment with topological structure, such as a sewer or street grid, which is made up of a set of interconnected locations. Fig. 1(a) shows six possible worlds, where solid lines indicate walls, dotted lines represent halls, and the circle marks the robot’s location. At time zero, the robot does not know the structure or size of its environment, but it does have a prior distribution over how many locations are possible and how they might be connected. It gets local observations indicating whether there are locations next to it, but these observations can be noisy: the robot might fail to see some openings. It can also execute actions by trying, sometimes unsuccessfully, to move to neighboring locations. Although this problem seems simple, it presents a challenging filtering task because there are infinitely many possible mazes. 1 Problem and Representation We will now define a filtering problem based on a logical representation of states, actions, transitions, and observations. This representation is inspired by a similar one previously used for probabilistic rule learning in the fully observable setting [8]. Let s be the (possibly countably infinite) set of possible states, and o be the finite set of possible observations. Then let si ∈ s and oi ∈ o be a specific state and a specific observation at time i, s0:t be a sequence {s0, . . . , st} of t+1 states, and o1:t be a sequence {o1, . . . , ot} of t observations. The filtering problem is to compute the distribution p(st|o1:t) for a sequence of time steps t = 1 . . . T . The dynamics of the world are p(s0:t, o1:t) = p(s0) t ∏ i=1 p(si|si−1)p(oi|si) . Logical Particle Filtering 3